Potential impact of annual vaccination with reformulated COVID-19 vaccines: lessons from the U.S. COVID-19 Scenario Modeling Hub (preprint)

ImportanceCOVID-19 continues to cause significant hospitalizations and deaths in the United States. Its continued burden and the impact of annually reformulated vaccines remain unclear. ObjectiveTo project COVID-19 hospitalizations and deaths from April 2023-April 2025 under two plausible assumptions about immune escape (20% per year and 50% per year) and three possible CDC recommendations for the use of annually reformulated vaccines (no vaccine recommendation, vaccination for those aged 65+, vaccination for all eligible groups). DesignThe COVID-19 Scenario Modeling Hub solicited projections of COVID-19 hospitalization and deaths between April 15, 2023-April 15, 2025 under six scenarios representing the intersection of considered levels of immune escape and vaccination. State and national projections from eight modeling teams were ensembled to produce projections for each scenario. SettingThe entire United States. ParticipantsNone. ExposureAnnually reformulated vaccines assumed to be 65% effective against strains circulating on June 15 of each year and to become available on September 1. Age and state specific coverage in recommended groups was assumed to match that seen for the first (fall 2021) COVID-19 booster. Main outcomes and measuresEnsemble estimates of weekly and cumulative COVID-19 hospitalizations and deaths. Expected relative and absolute reductions in hospitalizations and deaths due to vaccination over the projection period. ResultsFrom April 15, 2023-April 15, 2025, COVID-19 is projected to cause annual epidemics peaking November-January. In the most pessimistic scenario (high immune escape, no vaccination recommendation), we project 2.1 million (90% PI: 1,438,000-4,270,000) hospitalizations and 209,000 (90% PI: 139,000-461,000) deaths, exceeding pre-pandemic mortality of influenza and pneumonia. In high immune escape scenarios, vaccination of those aged 65+ results in 230,000 (95% CI: 104,000-355,000) fewer hospitalizations and 33,000 (95% CI: 12,000-54,000) fewer deaths, while vaccination of all eligible individuals results in 431,000 (95% CI: 264,000-598,000) fewer hospitalizations and 49,000 (95% CI: 29,000-69,000) fewer deaths. Conclusion and RelevanceCOVID-19 is projected to be a significant public health threat over the coming two years. Broad vaccination has the potential to substantially reduce the burden of this disease. Key pointsO_ST_ABSQuestionC_ST_ABSWhat is the likely impact of COVID-19 from April 2023-April 2025 and to what extent can vaccination reduce hospitalizations and deaths? FindingsUnder plausible assumptions about viral evolution and waning immunity, COVID-19 will likely cause annual epidemics peaking in November-January over the two-year projection period. Though significant, hospitalizations and deaths are unlikely to reach levels seen in previous winters. The projected health impacts of COVID-19 are reduced by 10-20% through moderate use of reformulated vaccines. MeaningCOVID-19 is projected to remain a significant public health threat. Annual vaccination can reduce morbidity, mortality, and strain on health systems.

Informing pandemic response in the face of uncertainty. An evaluation of the U.S. COVID-19 Scenario Modeling Hub (preprint)

Our ability to forecast epidemics more than a few weeks into the future is constrained by the complexity of disease systems, our limited ability to measure the current state of an epidemic, and uncertainties in how human action will affect transmission. Realistic longer-term projections (spanning more than a few weeks) may, however, be possible under defined scenarios that specify the future state of critical epidemic drivers, with the additional benefit that such scenarios can be used to anticipate the comparative effect of control measures. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make 6-month ahead projections of the number of SARS-CoV-2 cases, hospitalizations and deaths. The SMH released nearly 1.8 million national and state-level projections between February 2021 and November 2022. SMH performance varied widely as a function of both scenario validity and model calibration. Scenario assumptions were periodically invalidated by the arrival of unanticipated SARS-CoV-2 variants, but SMH still provided projections on average 22 weeks before changes in assumptions (such as virus transmissibility) invalidated scenarios and their corresponding projections. During these periods, before emergence of a novel variant, a linear opinion pool ensemble of contributed models was consistently more reliable than any single model, and projection interval coverage was near target levels for the most plausible scenarios (e.g., 79% coverage for 95% projection interval). SMH projections were used operationally to guide planning and policy at different stages of the pandemic, illustrating the value of the hub approach for long-term scenario projections.

Inter-rater reliability of the Infectious Disease Modeling Reproducibility Checklist (IDMRC) as applied to COVID-19 computational modeling research (preprint)

Background: Infectious disease computational modeling studies have been widely published during the coronavirus disease 2019 (COVID-19) pandemic, yet they have limited reproducibility. Developed through an iterative testing process with multiple reviewers, the Infectious Disease Modeling Reproducibility Checklist (IDMRC) enumerates the minimal elements necessary to support reproducible infectious disease computational modeling publications. The primary objective of this study was to assess the reliability of the IDMRC and to identify which reproducibility elements were unreported in a sample of COVID-19 computational modeling publications. Methods: Four reviewers used the IDMRC to assess 46 preprint and peer reviewed COVID-19 modeling studies published between March 13th, 2020, and July 31st, 2020. The inter-rater reliability was evaluated by mean percent agreement and Fleiss' kappa coefficients. Papers were ranked based on the average number of reported reproducibility elements, and average proportion of papers that reported each checklist item were tabulated. Results: Questions related to the computational environment (mean = 0.90, range = 0.90-0.90), analytical software (mean = 0.74, range = 0.68-0.82), model description (mean = 0.71, range = 0.58-0.84), model implementation (mean = 0.68, range = 0.39-0.86), and experimental protocol (mean = 0.63, range = 0.58-0.69) had moderate or greater (kappa > 0.41) inter-rater reliability. Questions related to data had the lowest values (mean = 0.37, range = 0.23-0.59). Reviewers ranked similar papers in the upper and lower quartiles based on the proportion of reproducibility elements each paper reported. While over 70% of the publications provided data used in their models, less than 30% provided the model implementation. Conclusions: The IDMRC is the first comprehensive, quality-assessed tool for guiding researchers in reporting reproducible infectious disease computational modeling studies. The inter-rater reliability assessment found that most scores were characterized by moderate or greater agreement. These results suggests that the IDMRC might be used to provide reliable assessments of the potential for reproducibility of published infectious disease modeling publications. Results of this evaluation identified opportunities for improvement to the model implementation and data questions that can further improve the reliability of the checklist.

Impact of SARS-CoV-2 vaccination of children ages 5-11 years on COVID-19 disease burden and resilience to new variants in the United States, November 2021-March 2022: a multi-model study (preprint)

Background: SARS-CoV-2 vaccination of persons aged 12 years and older has reduced disease burden in the United States. The COVID-19 Scenario Modeling Hub convened multiple modeling teams in September 2021 to project the impact of expanding vaccine administration to children 5-11 years old on anticipated COVID-19 burden and resilience against variant strains. Methods: Nine modeling teams contributed state- and national-level projections for weekly counts of cases, hospitalizations, and deaths in the United States for the period September 12, 2021 to March 12, 2022. Four scenarios covered all combinations of: 1) presence vs. absence of vaccination of children ages 5-11 years starting on November 1, 2021; and 2) continued dominance of the Delta variant vs. emergence of a hypothetical more transmissible variant on November 15, 2021. Individual team projections were combined using linear pooling. The effect of childhood vaccination on overall and age-specific outcomes was estimated by meta-analysis approaches. Findings: Absent a new variant, COVID-19 cases, hospitalizations, and deaths among all ages were projected to decrease nationally through mid-March 2022. Under a set of specific assumptions, models projected that vaccination of children 5-11 years old was associated with reductions in all-age cumulative cases (7.2%, mean incidence ratio [IR] 0.928, 95% confidence interval [CI] 0.880-0.977), hospitalizations (8.7%, mean IR 0.913, 95% CI 0.834-0.992), and deaths (9.2%, mean IR 0.908, 95% CI 0.797-1.020) compared with scenarios where children were not vaccinated. This projected effect of vaccinating children 5-11 years old increased in the presence of a more transmissible variant, assuming no change in vaccine effectiveness by variant. Larger relative reductions in cumulative cases, hospitalizations, and deaths were observed for children than for the entire U.S. population. Substantial state-level variation was projected in epidemic trajectories, vaccine benefits, and variant impacts. Conclusions: Results from this multi-model aggregation study suggest that, under a specific set of scenario assumptions, expanding vaccination to children 5-11 years old would provide measurable direct benefits to this age group and indirect benefits to the all-age U.S. population, including resilience to more transmissible variants.

Projected resurgence of COVID-19 in the United States in July-December 2021 resulting from the increased transmissibility of the Delta variant and faltering vaccination (preprint)

What is already known about this topic?The highly transmissible SARS-CoV-2 Delta variant has begun to cause increases in cases, hospitalizations, and deaths in parts of the United States. With slowed vaccination uptake, this novel variant is expected to increase the risk of pandemic resurgence in the US in July--December 2021. What is added by this report?Data from nine mechanistic models project substantial resurgences of COVID-19 across the US resulting from the more transmissible Delta variant. These resurgences, which have now been observed in most states, were projected to occur across most of the US, coinciding with school and business reopening. Reaching higher vaccine coverage in July--December 2021 reduces the size and duration of the projected resurgence substantially. The expected impact of the outbreak is largely concentrated in a subset of states with lower vaccination coverage. What are the implications for public health practice?Renewed efforts to increase vaccination uptake are critical to limiting transmission and disease, particularly in states with lower current vaccination coverage. Reaching higher vaccination goals in the coming months can potentially avert 1.5 million cases and 21,000 deaths and improve the ability to safely resume social contacts, and educational and business activities. Continued or renewed non-pharmaceutical interventions, including masking, can also help limit transmission, particularly as schools and businesses reopen.

TextEssence: A Tool for Interactive Analysis of Semantic Shifts Between Corpora (preprint)

Embeddings of words and concepts capture syntactic and semantic regularities of language; however, they have seen limited use as tools to study characteristics of different corpora and how they relate to one another. We introduce TextEssence, an interactive system designed to enable comparative analysis of corpora using embeddings. TextEssence includes visual, neighbor-based, and similarity-based modes of embedding analysis in a lightweight, web-based interface. We further propose a new measure of embedding confidence based on nearest neighborhood overlap, to assist in identifying high-quality embeddings for corpus analysis. A case study on COVID-19 scientific literature illustrates the utility of the system. TextEssence is available from https://github.com/drgriffis/text-essence.